Docs: Add warning about snapshot_ids arg in expired_snapshots procedure
#12291
Conversation
docs/docs/spark-procedures.md
Outdated
| @@ -270,7 +270,8 @@ the `expire_snapshots` procedure will never remove files which are still require | |||
| | `retain_last` | | int | Number of ancestor snapshots to preserve regardless of `older_than` (defaults to 1) | | |||
| | `max_concurrent_deletes` | | int | Size of the thread pool used for delete file actions (by default, no thread pool is used) | | |||
| | `stream_results` | | boolean | When true, deletion files will be sent to Spark driver by RDD partition (by default, all the files will be sent to Spark driver). This option is recommended to set to `true` to prevent Spark driver OOM from large file size | | |||
| | `snapshot_ids` | | array of long | Array of snapshot IDs to expire. | | |||
| | `snapshot_ids` | | array of long | Array of snapshot IDs to expire (note that the table's expiration properties will still be applied to remove all expired snapshots, unless `older_than` or `retain_last` arguments are also given). | | |||
There was a problem hiding this comment.
Not sure this note is clearer. I'm thinking about words like Additional array of snapshots IDs to expire, besides those expired by older_thanandretain_last arguments, and table properties.
There was a problem hiding this comment.
Thanks for the quick feedback! Reworded.
estherbester
force-pushed
the
expire-docs
branch
from
44a6225 to
fcd5757
Compare
|
I lost some context here, I remember there was a devlist discussion around this. could you link that to the PR? |
|
Would be helpful to add the specific code path for |
estherbester
force-pushed
the
expire-docs
branch
from
fcd5757 to
5d0b5e7
Compare
|
@kevinjqliu thanks for the feedback! I thought it might be more useful to link to the full class to show in context why the argument works the way it currently does. Also wasn't sure where exactly the links would go -- I didn't want to introduce too much inconsistency with the rest of the documentation. |
For context: https://apache-iceberg.slack.com/archives/C05BXHPEGTA/p1738897913969059